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ABSTRACT 


The  objective  of  this  thesis  is  to  design,  conduct,  and 
analyze  a  command,  control,  and  communications  conflict 
simulation  to  investigate  the  effects  of  differing  types  and 
quantities  of  information  delay  on  mission  outcomes  using  the 
computer  simulation  game  Tactical  Tic  Tac-Toe  (T4).  Three 
different  types  of  delay  were  compared:  tactical,  area,  and 
communications.  Each  type  of  delay  was  delayed  from  zero  to 
nine  moves.  The  results  indicate  that  tactical  delay  had  the 
greatest  effect  on  mission  outcome.  Area  delay  had  less  of  an 
impact.  Communications  delay  had  the  least  effect.  Contrary 
to  predictions,  within  each  type  of  delay,  different  levels  of 
delay  did  not  significantly  effect  mission  outcomes.  This  may 
be  attributed  to  the  high  variability  of  the  game  scores. 
Generally,  tactical  and  area  delays  graphically  showed  less 
mission  impact  at  lower  levels  of  delay,  however  communication 
delay  indicated  no  trend  in  mission  outcome  at  different 
levels  of  delay. 
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I.  INTRODUCTION 

A.  BACKGROUND 

The  study  of  Command,  Control,  and  Communications  (C3) 
places  a  large  emphasis  on  the  importance  of  information  flow 
in  combat  operations.  Considerable  work  has  been  done  to 
define  the  C3  process.  John  Boyd  developed  the  basic  Observe, 
Orient,  Decide,  and  ACT  (O-O-D-A)  C3  structured  model. 
Similarly,  Dr  Joel  Lawson  developed  his  Command  and  Control 
( C2 )  process  model  using  Sense,  Process,  Compare,  Decide,  and 
Act.  In  both  the  Boyd  and  Lawson  models,  information 
gathering  is  the  first  of  a  series  of  step-wise  processes  to 
define  C3 . 

Obviously,  the  information  used  in  the  C3  process  must  be 
timely  and  accurate  to  successfully  accomplish  a  mission. 
There  are  many  examples  in  military  history  which  show  the 
disastrous  effects  of  late  and/or  inaccurate  information  being 
used  in  the  decision  process.  One  example  is  the  attack  on 
the  U.S.S.  Liberty  in  1967  where  a  delay  in  communications 
resulted  in  the  ship  being  in  the  wrong  location  and 
mistakenly  attacked  by  Israel.  The  attack  killed  34,  wounded 
75,  and  destroyed  the  ship  [Ref.  l:p.  10].  It  is  possible  to 
investigate  past  military  conflicts  to  determine  the  effect  of 
information  quality  on  the  battle  outcome.  However,  because 
of  numerous  other  variables  for  each  case,  it  would  be 
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difficult  to  quantify  a  relationship  between  information 
quality  and  battle  outcome. 

An  alternative  to  historical  research  is  to  use  war  games 
and  simulation  which  have  been  used  effectively  to  investigate 
C3  systems.  War  games  use  human  subjects  to  make  decisions 
where  as  in  simulation,  decisions  are  made  according  to  a  set 
of  predetermined  rules  [Ref.  2:p.  6].  This  thesis  uses  a 
computer  simulation  to  investigate  the  effects  of  information 
delay  on  mission  performance  in  a  simulated  environment. 


B .  OBJECTIVE 

The  objective  of  this  thesis  is  to  design,  conduct,  and 
analyze  a  C3  conflict  simulation  to  investigate  the  effects  of 
differing  types  and  quantities  of  information  delay  on  mission 
outcome  using  the  computer  simulation  game  Tactical  Tic-Tac- 
Toe  (T4 ) .  Three  different  types  of  information  delay  (area, 
communications,  and  tactical)  are  used  in  T4 .  Each  type  of 
delay  can  be  set  from  zero  up  to  nine  moves. 

1.  Research  Questions 

Two  research  questions  are  to  be  answered: 

•  Of  the  three  types  of  T4  delays  (area,  communications, 
and  tactical) ,  which  one  has  the  greatest  effect  on  mission 
outcome? 

•  As  information  delays  increase  for  each  of  the  three 
types  of  delay,  is  there  a  point  at  which  the  mission  outcomes 
do  not  continue  to  decline  significantly  as  the  delay 
increases? 
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2.  Predictions 


The  outcome  of  simulated  conflicts  in  T4  not  only 
depends  on  the  type  and  amount  of  delay  but  also  on  several 
other  parameters  which  are  programmed  into  the  game.  However, 
assuming  the  experiment  is  designed  to  neutralize  the  effects 
of  collateral  parameters  and  to  isolate  the  effects  of 
information  delay,  the  following  predictions  were  made: 

•  Tactical  delay  would  have  the  most  effect  on  mission 
outcome . 

•  Delays  above  three  moves  would  have  little  effect  on 
mission  outcome  for  each  of  the  three  types  of  delay. 

Tactical  delay  is  the  delay  of  intelligence  on  the 
enemy's  position  on  the  player's  side  of  the  board.  The  first 
prediction  was  made  assuming  that  a  delay  in  a  player's 
knowledge  of  the  opponent  directly  opposing  him  would  have  the 
greatest  effect  on  the  mission  outcome.  Area  delay  is  a  delay 
of  intelligence  on  the  enemy's  position  on  the  other  side  of 
the  board.  Communication  delay  is  a  delay  of  information 
between  players  on  the  same  team.  Both  area  and  communication 
delay  were  predicted  to  effect  the  mission  outcome  less  than 
tactical  delay. 

The  second  prediction  was  made  based  on  the  fact  that 
T4  uses  pattern  recognition  in  computer  simulation  play.  The 
computer  sequentially  looks  at  all  eight  of  the  ways  that  a 
three  cell  TTT  can  be  scored  and  tries  to  match  the  three  cell 
pattern  with  the  game  plan  to  determine  a  weighting  factor. 
It  is  assumed  that  if  information  is  delayed  three  moves,  the 
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pattern  recognition  system  will  be  severely  effected  and 
delays  of  more  t'.’^n  three  will  have  marginally  less  impact  on 
the  outcome.  The  plot  shown  in  Graph  1  is  a  prediction  of 
mission  success  when  increasing  information  delay  is  imposed 
on  an  opponent. 


PREDICTED  MISSION  OUTCOME 
WHEN  INFORMATION  DELAY  IS 
IMPOSED  ON  THE  ENEMY 


01234567  8  9 

DELAY 

Graph  1 


Note  that  at.  rero  delay,  mission  success  is  predicted  to  be  50 
percent;  each  side  would  have  an  equal  chance  of  winning.  At 
delays  of  3  and  more,  an  almost  100  percent  mission  success  is 
predicted. 

C.  TACTICAL  TIC-TAC-TOE  (T4) 

T4  is  a  simple  board  game  based  on  the  well  known  game  of 
Tic-Tac-Toe  (TTT) .  It  was  developed  by  Mr  Gary  Porter,  a 
former  student  and  instructor  at  the  Naval  Postgraduate 
School.  T4  has  been  used  in  operational  science  courses  and 
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I 

command  and  control  courses  to  instruct  students  in  designing, 
conducting,  and  analyzing  system  experiments. 

T4  can  be  played  in  three  different  ways.  First,  it  can 
be  played  with  individual  players  opposing  each  other. 

Second,  a  team  of  individuals  can  play  the  computer.  Finally, 

T4  can  simulate  all  players  in  the  automated  version  of  the 
game.  Only  the  automated  version  was  used  for  this  thesis. 

T4  uses  two  tic-tac-toe  boards  with  two  teams,  X  and  0. 

Each  team  has  two  players,  a  left  side  and  right  side,  for  a 
total  of  four  players  a  game  as  shown  in  Figure  1. 


O  Left  0  Right 


X  Left  X  Right 


Figure  1.  T4  Game  Board 

Each  player  moves  only  on  their  side  of  the  board  to  score 
TTTs .  Missions  can  be  assigned  to  allow  scoring  of  TTTs 
across  game  boards  (crossover  TTTs)  encouraging  players  to 
gain  information  about  their  partner's  and  opponent's 
positions  on  the  other  board  side.  Each  player  moves 
simultaneously.  If  players  move  into  the  same  cell  on  the 
same  turn,  conflict  resolution  processes  occur  depending  on 
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how  the  game  is  set-up.  The  first  same  turn  conflict  can  be 
decided  randomly  or  awarded  to  a  particular  team.  Subsequent 
same  turn  conflicts  may  be  decided  either  randomly  or 
alternately.  Different  turn  conflicts  can  also  occur  when 
delays  are  introduced  into  the  game  because  a  player  may  try 
to  move  into  a  cell  that  appears  empty  to  the  player  but,  in 
fact,  is  already  occupied.  In  this  case,  the  first  player  in 
the  cell  keeps  it  and  the  other  player  losses  the  move. 

In  addition  to  the  type  of  delay,  amount  of  delay,  and 
conflict  resolution  types,  several  other  parameters  can  be 
varied  in  the  game.  Missions  can  be  set  up  as  victory  or 
survival.  A  victory  mission  requires  a  team  to  score  more 
TTTs  than  the  other  team  in  the  assigned  mission  (game  board) 
area  (either  left,  right,  crossover,  or  entire  game  board) . 
Survival  missions  require  a  team  to  score  as  many  or  more  TTTs 
as  the  opposition  in  the  assigned  mission  area.  Also  in  the 
automated  mode,  the  game  play  recognition  system  is  used  which 
requires  a  data  file  describing  simulated  player  personalities 
(offensive,  defensive,  passive,  etc.).  By  using  different 
data  files,  the  automated  player  can  play  different 
strategies.  A  detailed  set  of  T4  instructions  are  included  in 
Appendix  A. 

Chapter  II  covers  the  selection  of  the  parameters  and  the 
procedure  used  in  T4  to  investigate  the  two  research 
questions.  The  resulting  data  and  analysis  are  provided  in 
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Chapter  III.  Chapter  IV  contains  the  interpretation  of  the 
results  and  recommendations  for  future  T4  studies. 
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II.  EXPERIMENTAL  DESIGN 

A.  OVERVIEW 

The  automated  version  of  T4  was  used  to  investigate  the 
two  thesis  research  questions.  The  general  approach  was  to 
perform  multiple  game  repetitions  for  each  type  and  amount  of 
delay.  Only  one  parameter  at  a  time  was  changed  for  each  set 
of  replications.  Differing  types  and  levels  of  delay  were 
applied  to  the  O  team.  The  X  team  was  the  control  team.  Both 
0  team  players  were  set  for  the  same  delay  level  and  both  X 
players  were  always  set  for  zero  delay.  Multiple  delay  levels 
(two  or  more  delay  types  set  greater  than  zero)  were  not  used 
in  this  investigation.  Other  game  parameters  were  fixed  for 
all  replications.  This  chapter  covers  the  selection  of  the 
parameters  and  the  procedure  used  in  T4  to  investigate  the  two 
research  questions. 

B.  APPARATUS 

The  automated  T4  software  was  run  on  an  Apple  Macintosh 
computer  using  system  version  6.0.5  with  HyperCard  version  2.0 
software.  The  results  of  each  game  were  stored  in  Excel 
version  4.0  spreadsheet  software  operating  on  an  IBM  PC  type 
computer  using  Windows  version  3.1  operating  system.  Minitab 
PC  version  8  was  used  for  statistical  analysis. 
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C.  GAME  CONFIGURATIONS 


In  addition  to  the  type  and  amount  of  delay,  three  other 
parameters  must  be  set  in  the  game  plan  —  conflict 
resolution,  player  missions,  and  player  strategies.  Since  the 
type  and  amount  of  information  delay  were  to  be  investigated, 
it  was  necessary  to  carefully  select  and  fix  the  values  for 
the  other  three  parameters  so  the  effects  of  interest  could  be 
measured  in  the  game  results.  Figure  2  shows  the  game  board 
at  the  end  of  the  last  game  of  the  last  replication. 


This  view  of  the  game  board  has  the  game  plans  overlaid  on  it. 
It  shows  the  delay  level  and  the  configuration  of  the  other 
three  parameters.  Appendix  A  contains  additional  information 
on  the  configuration  parameters.  Except  for  the  type  and 
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amount  of  delay,  the  other  game  plan  parameters  remained  the 
same  for  all  replications. 

1.  Conflict  Resolution 

Conflict  resolution  for  same  turn  conflicts  was  set  to 
be  random  for  every  conflict.  This  provided  an  equal  chance 
to  both  the  X  and  0  sides  of  winning  conflicts  when  they 
occurred.  The  other  options  for  conflict  resolution  are:  (1) 
one  team  always  win  the  initial  conflict  then  teams  alternate 
for  the  subsequent  conflicts  wins;  (2)  one  team  always  wins 
the  first  conflict  then  randomly  select  which  team  wins  the 
subsequent  conflicts;  and  (3)  randomly  select  the  team  which 
wins  the  first  conflict  then  alternate  teams  for  the 
subsequent  conflicts. 

2.  Mission  Assignments 

Mission  assignments  for  both  sides  were  to  set  for 
victory  left  (VL)  and  victory  crossover  (VC) .  This 
combination  allowed  the  game  scores  to  reflect  both  one-sided 
play  and  crossover  play.  Also  by  setting  both  X  and  0  sides 
to  the  same  mission,  their  scores  could  be  easily  compared. 
The  total  possible  missions  for  each  side  are  victory  (V)  or 
survival  (S)  for  the  right  side  (R) ,  left  side  (L) ,  crossover 
(C) ,  and  overall  (0) .  Therefore,  a  total  of  eight  different 
missions  could  be  assigned  —  VR,  VL,  VC,  VO,  SR,  SL,  SC,  SO. 
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3.  Strategies 

To  run  the  automated  version  of  T4 ,  a  game  plan  file 
is  used  to  control  how  each  of  the  four  simulated  players 
plays  the  game.  Strategies  for  each  player  are  established  in 
a  set  of  three  game  plan  matrices  used  to  decide  each 
simulated  player's  move.  The  regular  matrix  controls  player 
moves  on  the  left  or  right  sides  of  the  board.  The  crossover 
matrix  controls  moves  in  the  crossover  area.  The  cell  matrix 
breaks  tries  and  determines  where  the  player  will  move  at  the 
start  of  the  game.  Together  the  three  matrices  specify  a 
simulated  player's  personality  to  be  offensive,  defensive, 
passive,  team  oriented,  lone  wolf,  random,  etc. 

An  offensive  crossover  (OC)  strategy  was  chosen  to 
achieve  the  VL  mission  while  a  defense  crossover  (DC)  strategy 
was  chosen  to  achieve  the  VC  mission.  OC  was  used  for  both 
the  XL  and  OL  players  and  DC  was  used  for  the  XR  and  OR 
players.  OC  strategy  encourages  the  simulated  players  to 
achieve  TTTs  on  both  their  sides  of  the  board  and  in  the 
crossover  area.  DC  encourages  blocking  opponent's  TTTs  at  the 
expense  of  offensive  scoring.  See  Appendix  A  for  detailed 
information  on  how  the  T4  program  uses  game  plan  matrices  to 
select  the  next  move. 
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D.  PROCEDURE 

Three  data  collection  files  were  created,  one  for  each 
type  of  delay  -  tactical,  area,  and  communications.  Each  file 
contains  results  of  30  replications  for  each  level  of  delay. 
There  were  3  types  of  delay  and  10  levels  of  delay,  each  of 
which  was  run  30  times  for  a  total  of  900  games  (3  x  10  x  30) . 
All  900  games  ran  sequentially.  The  replications  were  started 
on  13  Nov  92  at  1726  hours  and  were  completed  on  15  Nov  92  at 
1157  hours.  The  replications  ran  continuously  without 
interruption.  The  results  were  saved  as  text  files  (*.txt) 
then  transferred  to  an  Excel  spreadsheet  for  data  reduction. 
Each  of  the  3  spreadsheets  are  117  columns  by  300  rows. 
Appendix  B  contains  the  replication  set-up  files.  Selected 
data  was  transferred  to  Minitab  for  statistical  analysis  by 
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saving  data  in  Excel  using  Lotus  1-2-3  format  (*.WK1  files) 
and  then  retrieving  it  in  Minitab  using  the  Lotus  data  option. 
The  data  analysis  procedure  and  the  results  are  described  in 
Chapter  III. 
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III.  DATA  ANALYSIS 


A.  SCORING 

One  of  the  most  important  steps  in  system  investigations 
is  to  select  suitable  measures  of  effectiveness  (MOE)  so  the 
proper  conclusions  can  be  achieved.  Several  different  MOEs 
were  investigated  by  plotting  out  the  average  score  of  the  30 
repetitions  for  each  delay  set,  versus  the  amount  of  delay 
introduced  on  the  O  side.  The  total  score  for  the  O  side,  the 
difference  between  the  total  scores  for  the  X  and  O  sides,  and 
the  toi_dl  TTTs  were  looked  at  as  possible  scoring  measures. 
However,  the  difference  between  the  X  team  and  O  team  total 
TTTs  in  the  assigned  mission  areas  (VL  and  VC)  was  finally 
selected  as  the  MOE. 

The  total  score  is  the  sum  of  scores  for  each  mission 
assigned.  A  team  is  awarded  one  point  for  each  mission  that  is 
successfully  completed.  Since  two  missions  (VL  and  VC)  were 
assigned  and  they  were  same  for  each  side,  there  are  only  four 
different  game  score  outcomes:  2-0;  1-1;  1-0;  and  0-0. 
However,  if  scoring  is  measured  in  mission  TTTs,  as  many  as  18 
or  as  few  as  0  could  be  scored  per  side.  Consider  a  game 
where  the  X  side  beats  the  0  side  2-0  in  total  score.  The  X 
side  may  beat  the  0  side  by  one  TTT  in  each  mission  area;  or 
in  contrast,  the  X  side  my  beat  the  O  side  by  overwhelming 
margins  in  each  mission  area.  Although  the  total  score 


determines  the  wir  ..ing  team,  it  does  not  provide  much  insight 
about  the  margin  of  victory.  Mission  TTTs  provide  a  much 
finer  measurement  and  truer  indication  on  game  outcomes  as 
compared  to  the  total  score. 

By  introducing  a  delay  on  the  0  side,  it  was  hypothesized 
that  0's  mission  performance  would  decrease.  The  performance 
decrease  could  occur  either  offensively  (O  scoring  less  TTTs) , 
defensively  (X  scoring  more  TTTs) ,  or  a  combination  of  the 
two.  To  ensure  that  a  change  in  performance,  both  offensively 
and  defensively,  was  reflected  in  the  scoring  measure,  the 
difference  between  the  X  side  and  0  side  total  TTTs  in  the 
assigned  mission  areas  was  selected  as  the  scoring 
measurement.  Note  that  if  O's  mission  performance  decreases, 
the  scoring  value  will  increase  positively.  If  each  side  is 
configured  with  the  same  delay  and  game  plan,  then  near  equal 
performance  is  expected  and  the  scoring  values  should  be 
approximately  zero. 

B.  STATISTICAL  METHODS 

l.  Distribution 

The  average  of  TTTs  in  the  mission  areas  for  each  30 
game  repetition  set  was  assumed  to  have  a  normal  distribution 
by  the  Central  Limit  Theorem  (CLT)  of  statistics.  The  CLT 
states  that  the  averages  of  random  samples  taken  from  any  type 
of  distribution  will  approximate  a  normal  distribution  if  the 
size  of  the  samples  ( n )  are  sufficiently  large.  A  rule  of 
thumb  for  the  size  of  n  is  that  n  must  be  greater  than  30. 
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Since  30  repetitions  were  used  for  each  game  set,  it  was 
assumed  that  this  was  sufficient  to  justify  using  the  normal 
distribution  approximation.  Also,  since  the  scoring  measure 
was  the  difference  between  TTTs  for  the  X  and  O  sides  it  would 
also  be  normally  distributed  since  the  difference  is  a  linear 
combination  of  normally  distributed  variables. 

2.  Analysis  of  Variance 
a .  Single-Factor 

The  second  research  question:  is  there  a  point  at 
which  mission  outcomes  do  not  decline  significantly  as  the 
delay  increases,  was  investigated  using  single-factor  Analysis 
of  Variance  (ANOVA)  .  Each  of  the  three  types  of  delay 
(tactical,  area,  and  communication)  were  examined  separately. 
For  each  type  of  delay,  there  were  10  different  amounts  of 
delay  used  (J  =  10;  0  -  9)  and  30  repetitions  performed  (J  = 
30) .  To  answer  the  thesis  question,  the  null  hypothesis  must 
be  checked  to  compare  the  means: 

H0  :  Mo  =  Mi  =  ...  =  M 9 
versus 

Ha  :  at  least  two  of  the  m,'s  are  different. 

If  the  null  hypothesis  can  not  be  rejected,  then  none  of  the 
delays  resulted  in  significant  changes  in  the  scoring.  If 
this  is  the  case,  there  is  not  a  point  at  which  the  mission 
outcomes  do  not  significantly  continue  to  decline  as  delay 
increases.  However,  if  the  null  hypothesis  can  be  rejected, 
then  at  least  two  of  the  means  are  different.  Tukey's  method 
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can  then  be  used  to  determine  which  means  are  significantly 
different  and  possibly  determine  if  there  is  a  point  at  which 
mission  outcome  does  not  continue  to  decline. 
b .  Two-Factor 

To  answer  the  first  research  question:  which  of 
the  three  types  of  delay  has  the  greatest  effect  on  mission 
outcome,  two  factor  ANOVA  was  used.  The  first  ANOVA  factor 
(A)  is  the  type  of  delay  consisting  of  three  levels  (1=3;  i 
=  area,  tactical,  and  communication).  The  second  factor  (B) 
was  the  amount  of  delay  consisting  of  9  levels  (J=9;j= 
1,2...  9).  Delays  of  zero  were  not  used  in  this  analysis 
because  when  each  of  the  types  of  delay  are  set  to  zero,  the 
games  are  the  same  and  identical  results  are  expected.  There 
were  3  0  observations  for  each  of  the  IJ  combinations  ( Kt)  = 
30) .  The  following  experimental  model  was  used: 

M*  =M  +  a,  +  I 3j  +  7,y 
where: 

=  the  mean  score  of  when  I  =  i  and  J  =  j 
M  =  the  expected  score  averaged  over  all  levels  (the  true 
grand  mean) . 

a,  =  main  effects  for  type  of  delay  factor  when  J  =  i 
(3j  =  main  effects  for  amount  of  delay  factor  when  J  =  j 
7,y  =  interaction  parameter  when  I  =  i  and  J  =  j 
For  this  model  three  sets  of  hypotheses  and  alternate 
hypotheses  are  considered: 
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The  no-interaction  hypothesis: 

hoab  ■  y,j  =  0  for  all  i,j 
versus  :  at  least  one  ytJ  ^  0 

The  rain  effects  hypothesis  for  type  of  delay: 

Boa  :  aJ  =  a2  = 

versus  :  at  least  one  a,  ?£  0 

The  main  effects  hypothesis  for  amount  of  delay: 

Bob’  =  @2  =  •  •  •  =  @  9  =0 

versus  HaB  :  at  least  one  &  0 

First  the  no-interactive  hypothesis  must  be  tested  to  see  if 
the  interactive  effects  are  significant.  If  they  are  not 
significant,  an  additive  model  can  be  assumed  and  the  two 
main  effects  can  be  checked  for  significance.  If  interaction 
is  significant  then  graphical  methods  must  be  used  to 
interpret  the  interaction. 

The  main  effects  hypothesis  for  the  type  of  delay 
was  used  to  test  the  first  thesis  question  as  to  which  type  of 
delay  has  the  greatest  effect  on  mission  outcome.  If  the 
hypothesis  can  be  rejected,  then  at  least  one  of  the  three 
types  of  delay  is  significantly  different.  In  this  case, 
Tukey's  method  can  be  used  to  identify  which  type  of  delay  is 
significantly  different. 

3.  Level  of  Significance 

Traditional  levels  of  significance  are  0.10,  0.05,  and 
0.01.  Previous  statistical  analyses  on  T4  have  used  a  value 
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of  0.05  [Ref.  3 : p .  28].  This  value  was  also  used  throughout 
the  data  analysis  in  this  study. 

C.  RESULTS 

1.  Increasing  Delay  Amounts 

To  answer  the  second  research  question,  each  of  the 
delay  types  were  studied  independently  to  determine  if  there 
was  a  point  at  which  increasing  the  delay  amount  did  not 
effect  mission  outcome.  Each  type  of  delay  was  analyzed  for 
delay  amounts  from  zero  to  nine  using  the  single-factor  ANOVA 
procedure  in  Minitab.  The  Minitab  analysis  for  area, 
communications,  and  tactical  delay  is  provided  in  Appendix  C. 
The  scoring  results  for  each  game  and  the  statistics  for  the 
3  0  game  repetition  sets  are  provided  in  Appendix  D.  As 
expected,  delay  levels  of  zero  for  each  type  of  delay  resulted 
in  scores  of  approximately  zero, 
a.  Area  Delay 

Graph  2  shows  the  effect  of  increasing  area  delay 
on  mission  outcome. 
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AREA  MISSION  PERFORMANCE 


Graph  2 


Delays  of  zero  to  two  produced  scores  of  approximately  zero. 
Delays  above  three  exhibited  scores  around  0.9  except  at  7  and 
9  where  relatively  high  and  low  scores  occurred,  respectively. 
Single  factor  ANOVA  produced  a  p-value  of  0.122,  greater  than 
the  0.05  level  of  significance.  Therefore,  the  null 
hypothesis  can  not  be  rejected  and  the  delay  amounts  did  not 
significantly  effect  the  score. 

b.  Communications  Delay 

Graph  3  shows  the  effect  of  increasing 
communications  delay  on  mission  outcome. 
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COMMUNICATIONS  MISSION 
PERFORMANCE 


Graph  3 

Graphically  the  results  do  not  indicate  any  trend  with 
increasing  delay.  Single-factor  ANOVA  strongly  supports  this 
indication  with  a  p-value  of  0.991,  well  above  the  0.05  level 
of  significance.  The  null  hypothesis  can  not  be  rejected  and 
the  delay  amounts  did  not  significantly  effect  the  score, 
c.  Tactical  Delay 

Graph  4  shows  the  effect  of  increasing  tactical 
delay  on  mission  outcome. 
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TACTICAL  MISSION  PERFORMANCE 


Graph  4 


After  0  delay,  the  score  increased  to  approximately  1.75  and 
remained  at  about  that  level  except  at  delays  of  6  and  9  where 
the  score  was  approximately  2.7.  Single-factor  ANOVA  analysis 
produced  a  p-value  of  0.092  slightly  above  the  0.05  level  of 
significance.  The  null  hypothesis  can  not  be  rejected  and 
therefore  the  delay  amounts  did  not  significantly  effect  the 
score. 

2 .  Delay  Types 

The  two-factor  ANOVA  analysis  using  Minitab  was  used 
to  answer  the  first  research  question.  First,  the  no¬ 
interaction  hypothesis  was  tested  followed  by  the  two  main 
effects  hypotheses.  The  Minitab  results  are  provided  in 
Appendix  E. 

The  no-interaction  hypothesis  test  resulted  in  a  p- 
value  of  0.528,  greater  than  the  0.05  level  of  significance. 
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Therefore  H0AB  can  not  be  rejected  and  the  interaction  effects 
of  amount  and  type  of  delay  are  not  significant.  The  presence 
or  absence  of  the  main  effects  were  then  tested. 

The  main  effects  hypothesis  test  for  type  of  delay 
resulted  in  a  p-value  of  0.000,  less  than  the  0.05  level  of 
significance.  Therefore  HOA  was  rejected  and  the  type  of  delay 
does  effect  the  mission  outcome. 

To  decide  which  types  of  delay  are  significantly 
different,  Tukey's  method  was  used.  For  the  type  of  delay 
factor,  1=3,  so  with  0.05  level  of  significance  and  IJ (K  - 
1)  =  (3)  (9)  (30  -  1)  =  783,  2.05,3,783  =  3.31  .  Then  w  = 
£>(  {MSB/  ( JK )  )  1/2  =  3 . 31(8.226/  (  (9)  (30)  )) 1/2  =  0.578.  The  overall 
means  for  area,  communications,  and  tactical  were  0.70,  -0.11, 
and  2.05  respectively.  By  ordering  the  means,  -0.11,  0.70, 
2.05,  it  is  clear  that  the  difference  between  each  mean  is 
greater  than  0.578  and  therefore  each  delay  type  is 
significantly  different  from  the  other.  Tactical  delay  had 
the  greatest  effect  on  mission  outcome,  followed  by  area,  and 
communications  having  the  least  effect.  Graph  5  compares  the 
performance  of  the  three  types  of  delay. 
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The  main  effects  hypothesis  test  for  amount  of  delay  resulted 
in  a  p-value  of  0.395,  greater  than  the  0.05  level  of 
significance.  Therefore  H0B  can  not  be  rejected  and  the 
amount  of  delay  does  not  effect  the  mission  outcome.  Although 
this  was  not  a  specific  research  question,  the  results  are 
consistent  with  the  findings  for  the  second  research  question. 
Differing  delay  amounts  do  not  effect  game  outcomes 
significantly  within  each  type  of  delay  or  overall  as  the  main 
effects  hypothesis  indicates. 

Chapter  IV  summarizes  the  results  and  provides  some 
recommendations  for  future  T4  studies. 
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IV.  CONCLUSION 


A.  SUMMARY 

Comparison  of  information  types  and  levels  in  T4  produced 
the  following  results  to  the  two  original  thesis  questions: 

•  Of  the  three  types  of  delay,  tactical  delay  was  found  to 
have  the  greatest  effect  on  mission  outcome  followed  by  area 
delay  which  had  less  of  an  effect.  Communication  delay  did 
not  have  any  discernible  effect  on  mission  outcome. 

•  For  each  type  of  delay,  different  levels  of  delay  did  not 
significantly  effect  mission  outcome.  However,  area  delay  and 
tactical  delay  graphically  indicated  delay  levels  at  which 
mission  outcomes  did  not  continued  to  decline  as  delay  levels 
were  increased. 

As  predicted,  tactical  delay  had  the  greatest  effect  on 
mission  outcome;  however  contrary  to  predictions,  there  was 
not  a  significant  difference  in  the  different  levels  of  delay 
for  each  type  of  delay.  Both  area  delay  and  tactical  delay 
graphically  showed  a  tendency  not  to  continue  to  degrade 
mission  outcome  once  past  a  certain  delay  level  —  a  delay  of 
three  for  area  and  one  for  tactical.  However,  ANOVA  analysis 
indicated  no  significant  differences  in  mission  outcomes  for 
the  different  delay  levels.  This  can  be  attributed  to  the 
relatively  large  variability  in  the  data  as  indicated  by  the 
standard  deviations  of  the  scores  for  each  30  replication 
delay  set.  The  standard  deviations  ranged  from  2.67  to  3.68 
for  averages  ranging  from  -0.467  to  3.300. 
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Three  experimental  design  decisions  probably  contributed 
to  the  high  variability  of  the  data:  mission  assignments, 
conflict  resolution,  and  game  plans.  Each  team  was  assigned 
the  same  mission,  strategy,  and  the  same  set  of  game  plans 
which  simulated  their  personalities.  This  combination  could 
have  resulted  in  each  team  having  a  tendency  to  perform 
similar  moves  resulting  in  a  higher  number  of  same  turn 
conflicts.  Since  conflicts  were  decided  randomly,  more 
conflicts  may  have  increased  the  variability  in  the  game 
scores.  No  attempt  was  made  to  investigate  this  theory, 
although  the  data  is  available  for  future  work. 

B.  RECOMMENDATIONS 

One  change  in  T4  that  may  be  helpful  would  be  to  fill  in 
zeros  on  the  output  spreadsheet  for  cells  where  zero  scores 
occurred.  At  present,  T4  leaves  the  cells  empty  for  zero 
values.  If  scoring  statistics  are  calculated  using  the  Excel 
functions,  the  empty  cells  are  omitted  in  the  calculations 
resulting  in  incorrect  statistical  values.  This  was  noticed 
when  computing  the  30  replication  delay  set  averages.  Excel 
computed  sample  sizes  using  cells  with  values  and  omitted  the 
samples  with  zero  scores.  This  can  easy  be  worked  around  by 
using  the  Excel  replace  command  to  fill  in  the  empty  cells 
with  zeros;  however,  a  change  in  the  T4  software  to  fill  in 
zeros  would  save  a  step  in  the  data  analysis  process. 
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There  are  many  more  possibilities  for  future  studies  using 
T4 .  The  following  are  a  few  ideas: 

•  Investigate  the  effect  of  same  and  different  mission 
assignments  on  mission  outcomes. 

•  Investigate  the  effect  of  game  plans  on  mission  outcomes. 

•  Investigate  the  effect  of  same  turn  conflict  resolution 
on  mission  outcomes.  Also,  look  at  the  effects  of  the  very 
first  conflict  wins  for  each  game  on  mission  outcomes. 

•  Investigate  the  effect  of  communication  delay  in  greater 
detail.  Communication  delay  did  not  appear  to  have  any  effect 
on  mission  outcome  in  this  study. 

•  Experiment  with  multiple  delay  levels  (two  or  three  types 
of  delay  set  greater  than  zero  moves) .  Is  there  a  synergistic 
effect  on  mission  outcomes? 

T4  is  a  powerfully  simulation  tool  for  instructing 
students  in  designing,  conducting,  and  analyzing  system 
experiments.  The  game  has  many  variables  that  can  be  set  to 
investigate  the  influence  on  mission  outcomes.  Many  output 
parameters  are  also  available  to  use  as  possible  MOEs.  This 
study  attempted  to  provide  some  insight  on  the  effects  of 
information  delay  within  T4 .  There  are  many  more  challenging 
areas  available  for  future  studies. 
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APPENDIX  A 


T4  (TACTICAL  T3) 


The  T4  game  used  in  the  OS4602  course  last  fall  was  derived  from  T3  (Tic  Tac 
Toe).  This  paper  describes  the  evolution  of  the  T4  game,  its  variants  and  its  potential 
use  as  a  simple,  but  rich  experimental  testbed  for  future  OS4602  experiments.  Also 
provided  is  a  discussion  of  a  computer  based  aid  for  use  by  the  controllers  of  the 
experiment.  Use  of  the  aid  will  result  in  more  accurate  results  and  faster  (more)  trials. 


T3  (TIC  I  AC  TOE)  REVIEW 


Rules: 

•  2  Players  (X  and  O) 

•  Alternate  Turns  (X  plays  first) 

•  First  player  to  get  3  in  a  row  (TTT)  wins: 

-  Horizontal 

-  Vertical 

-  Diagonal 

•  Ties  (Cats  Games)  are  possible 

Examples: 


_0 

0 

12 

0 

X 

X 

/ 

X 

X 

0 

2 

0 

0 

X 

Score:  X  Wins  Score:  Tie 
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T4  BASELINE 

The  T4  baseline  game  is  like  except  for  the  following: 

•  Simultaneous  moves:  Both  players  choose  their  moves  in  secret  and  then 
announce  them  simultaneously. 

•  Conflict  resolution:  The  simultaneous  move  rule  means  that  two  players  can 
move  into  the  same  cell  on  the  same  move.  A  coin  is  used  to  resolve  the  conflict.  The 
winner  of  the  coin  toss  is  awarded  the  move.  The  loser  is  not  allowed  to  make  an 
alternate  move;  thus  losing  a  turn. 

•  Scoring:  The  player  with  the  most  I  i  is  wins  (not  the  first  I  I  I  ).  Therefore  the 
maximum  score  is  8  and  the  minimum  score  is  0.  Although  the  Pr(8  I  I  Is)  is  about  1  in 
20,000,000  (about  the  same  odds  as  winning  the  Lotto). 


Private  Move  Sheet 


Public  Board 
N 


W 


Private  Move  Sheet 


o 


Sequence  of  Play: 


1.  Write  move  on  private  move  sheet. 

2.  Simultaneously  announce  moves. 

3.  Resolve  conflicts. 

4.  Post  on  public  board. 
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T4  Baseline  Example: 


TURN  1: 

Private  Move  Sheet 


W 


X 

TURN  1A 
TURN  2; 

Private  Move  Sheet 


SE 


X 

TURN  2A 
TURN  n: 


Public  Board 
N 


0. 

* 

S 

TURN  IB 
Public  Board 


S 

TURN  2C 
Public  Board 


Last  Turn  A 


Last  Turn  B 


Private  Move  Sheet 


NE 


O 

TURN  1 A 


Private  Move  Sheet 


SE 


Conflict 

Resolution 


IS 

Flip  Coin 
(X  wins) 


TURN  2A  TURN  2B 


SCORE: 
X  =  2 
0=0 


Last  Turn  A 


T4  WITH  INTELLIGENCE  DELAY 

The  intelligence  delay  game  is  like  the  T4  baseline  game  except  an  enemy 
detection  delay  factor  is  introduced.  A  player's  knowledge  of  own  moves  is  always 
real  time,  however  the  player's  knowledge  of  the  enemy’s  current  move  may  be  kept 
secret  for  one  or  more  turns  depending  on  the  game  configuration  selected  bef-  re 
game  start.  For  example,  assume  player  X  is  configured  to  play  a  one  turn  intelligence 
delay  game  and  further  assume  that  two  moves  for  both  players  have  already 
occurred.  Then  Player  X  knows  of  own  moves  (Xi  and  X2)  but  is  only  provided  player 
O's  first  move  (0-j).  The  configurations  assigned  to  each  player  may  be  different.  In 
the  above  example,  player  0  might  have  been  assigned  no  intelligence  delay  and 
therefore  would  know  of  all  four  moves  ( X-j ,  X2  Oi  anc*  O2). 


Games  with  intelligence  time  delays  can  result  in  cases  where  a  rational  player 
will  move  into  a  cell  already  occupied  by  the  enemy  (but  because  of  the  delay  factor, 
the  player  is  unaware  of  the  enemy's  location).  The  conflict  is  resolved  by  awarding 
the  move  to  the  player  first  occupying  the  square  (LIFO).  Players  are  notified  of  the 
conflict  and  of  the  resolution  on1  a  real  time  basis.  T4  with  intelligence  delays  requires 
maintaining  a  private  game  board  for  each  player  and  either  an  umpire  or  opponents 
who  are  good  sports. 


DOUBLE  BOARD  T4  BASELINE 


This  version  uses  two  standard  3X3  celi  boards  placed  side-by-side 
designated  left  and  right  game  boards.  Thus,  the  total  game  board  is  3  X  6  cells. 
This  game  is  like  the  T4  baseline  game  except: 

•  4  Simultaneous  Moves:  Both  players  announce  their  left  and  right  moves 
simultaneously. 

•  Crossover  Scores:  A  crossover  score  is  a  TTT  that  crosses  over  the  center  line 
between  the  left  and  right  game  boards.  A  crossover  score  can  include  TTTTTT  (e.g.; 
X  wins  the  top  row  of  both  the  left  and  right  game  boards.  Diagonal  crossover  TTTs 
are  also  possible,  but  vertical  TTTs  are  not.  Multiple  TTTs  are  scored  by  counting  the 
number  of  TTTs  embedded  in  a  string  of  multiple  TTTs.  l.e; 


TTT  =  1  TTT 
TTTT  s  2  TTTs 
TTTTT  =  3  TTTs 
TTTTTT  =  4  TTTs 

The  maximum  possible  score  is  26,  minimum  score  is  0. 


N 


” 

Left  < 

;  Right  j 

4 


DOUBLE  BOARD  T4  WITH  INTELLIGENCE  DELAY 


This  version  combines  the  rules  of  the  single  board  T4  intelligence  delay  game 
with  the  double  board  T4  baseline  game.  Again  the  enemy  detection  delay 
configuration  can  be  set  to  real  time  {no  delay),  or  a  delay  of  one  or  more  turns.  A 
different  delay  factor  can  be  chosen  for  each  player,  for  each  side  of  the  board,  or  in 
combinations.  For  example  one  time  delay  configuration  might  allow  player  X  to 
received  real  time  intelligence  on  the  left  side  of  the  board  but  never  receive 
intelligence  on  the  right  side.  While  during  the  same  game  opponent  player  O 
receives  intelligence  delayed  by  one  turn  on  both  sides  of  the  board. 


DOUBLE  BOARD  T4  WITH  MISSIONS 

Missions  are  assigned  to  achieve  an  outcome  (victory  or  survival)  within  an 
area  of  the  game  board  (left,  right,  crossover,  or  overall).  Victory  is  achieved  by 
scoring  more  TTTs  than  the  enemy  in  the  assigned  mission  area.  Survival  is  achieved 
by  not  losing  (i.e.;  winning  or  tieing)  the  assigned  mission  area.  The  eight  mission 
assignments  are  as  follows: 

LV  =  Left  side  victory:  Score  the  most  TTTs  on  the  left  side. 

RV  =  Right  side  victory:  Score  the  most  TTTs  on  the  right  side. 

CV  =  Crossover  victory:  Score  the  most  crossover  TTTs. 

OV  =  Overall  victory:  Score  the  highest  total  TTTs  (left,  right,  and  crossovers  ). 

LS  =  Left  side  survival:  don't  lose  on  the  left  side. 

RS  s  Right  side  survival:  don't  lose  on  the  right  side. 

CS  =  Crossover  survival:  don't  lose  in  the  crossover  area. 

OS  =  Overall  survival:  don't  lose  overall. 

/ 

Up  to  four  individual  missions  may  be  assigned  to  a  player's  mission  set.  Both 
players  can  be  assigned  different  mission  sets.  Scoring  is  based  on  successful 
mission  achievement  (either  by  individual  missions  or  by  mission  set).  TTTs  are  used 
to  decide  mission  outcome.  Notice  that  non-zero  sum  games  are  possible.  I.e.;  both 
sides  may  achieve  a  degree  of  success  in  some  game  configurations. 
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DOUBLE  BOARD  T4  WITH  MISSIONS  AND  INTELLIGENCE  DELAY 


A  combination  of  the  double  board  T4  baseline  game  with  mission  assignments 
(one  or  more  of  eight  mission  areas)  and  intelligence  delay  (real  time,  or  delays  of  one 
or  more  turns. 

TEAM  GAMES 

Team  games  consist  of  two  players  per  side:  One  player  on  a  team  is  assigned 
the  left  game  board  and  the  other  the  right.  Team  games  are  always  played  on  double 
game  boards  with  mission  assignments.  Missions  (including  mission  sets)  are 
assigned  by  team.  I.e.;  the  players  on  a  team  are  always  assigned  the  same  missions. 
However  different  mission  sets  may  be  assigned  to  the  opposing  team.  As  a  general 
rule  team  games,  especially  combinations  of  games  discussed  below,  require  an 
umpire:  Someone  who  administers  the  game  to  ensure  that  the  appropriate  amount  of 
information  is  provided  to  all  four  players.  As  discussed  below,  players  on  the  same 
team  may  be  assigned  different  delay  factors.  Thus  five  private  game  boards  are 
required:  one  for  each  player  and  the  umpire.  All  players  keep  their  game  board 
secret  from  opponents  and  their  own  partner.  The  umpire's  view  of  the  game 
represents  ground  truth  and  is  therefore  also  kept  secret.  Players  secretly  provide 
their  moves  to  the  umpire.  The  umpire  then  secretly  updates  ground  truth,  resolves 
conflicts,  and  then  secretly  returns  the  appropriate  level  of  information  to  each  player. 


TEAM  GAMES  WITH  INTELLIGENCE  DELAY 

The  intelligence  delay  factor  controls  when  players  are  provided  the  enemy's 
moves.  Intelligence  delay  factors  may  be  assigned  independently  of  player  and  game 
board  side.  Define  Tactical  intelligence  to  be  enemy  position  information  on  the 
player's  side  of  the  board,  and  Area  intelligence  to  be  enemy  position  information  on 
the  other  side  of  the  board.  Then  different  tactical  and  area  intelligence  delay  factors 
may  be  assigned  each  of  the  four  players.  E.g.;  X  left  may  be  assigned  a  tactical 
intelligence  delay  factor  of  1  turn  and  an  area  intelligence  delay  factor  of  2  turns  (Xj_  = 
T i ,  A2).  The  players  partner  (Xj_)  and  the  enemies  (0[_,  Op)  may  each  be  assigned 
different  intelligence  delay  factors  . 
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TEAM  GAMES  WITH  COMMUNICATIONS  DELAYS 


This  version  introduces  a  communications  delay  factor  between  players  on  the 
same  team  (partners).  Communications  delay  controls  the  timeliness  of  receiving  your 
partners  move  information.  Like  intelligence  delay  factors,  communications  delays 
may  be  set  to  real  time  (no  delay)  or  a  delay  of  one  or  more  turns.  Continuing  the 
example,  if  player  X|_  is  assigned  a  communications  delay  of  one  turn  then  Xj_  =T1, 
A£,  Ci .  Each  of  the  four  players  may  be  assigned  different  communications  delays. 

TEAM  GAMES  WITH  FEEDBACK  DELAYS 

This  version  specifies  the  number  of  moves  each  player  must  make  before 
receiving  feedback  on  enemy  and  partner  locations.  Like  the  three  other  delay  factors 
the  feedback  delay  can  be  set  from  no  feedback  delay  to  more  than  one  turn  feedback 
delay.  In  the  no  feedback  delay  configuration,  normal  feedback  is  provided  after  each 
turn.  Where  normal  means  providing  the  level  of  information  dictated  by  the  other 
delay  factors  (tactical  intelligence,  area  intelligence,  and  communications  delays).  If 
the  delay  factor  is  greater  than  one,  then  multiple  moves  are  required  before  normal 
feedback  is  provided.  E.g.;  if  the  feedback  delay  is  set  to  2,  then  the  player  must  make 
two  moves  before  receiving  position  information  on  other  players'  moves.  In  this 
configuration,  information  when  received  is  current  to  the  second  turn  (after  adjusting 
for  intelligence  and  communications  delays).  Each  of  the  four  players  may  be 
assigned  different  feedback  delays. 


TEAM  GAMES  WITH  PLANNING 

This  version  introduces  planning  constraints  between  partners.  Planning  is  that 
part  of  the  game  that  occurs  before  the  first  move  is  executed.  There  are  three  levels 
of  planning.  Planning  levels  assigned  to  partners  must  be  the  same.  However  the 
levels  assigned  opposing  teams  may  be  different. 

•  Specific  scenario  planning  allowed.  Scenarios  are  provided  and  direct 
conversation  is  allowed  between  partners  before  game  start  in  order  to  plan  the 
specific  mission. 
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•  General  planning  allowed.  Team  members  are  assigned  and  genera!  planning 
sessions  are  allowed.  Planning  if  not  permitted  after  the  specific  game  scenario 
(factors)  are  assigned. 

•  Planning  not  allowed.  Players  are  assigned  to  teams  from  the  player  pool.  No 
team  conversation  is  permitted  until  the  end  of  the  game. 


EXPERIMENTAL  DESIGN  COMBINATIONS  FOR  TEAM  GAMES 

Six  factors  each  having  multiple  treatment  levels  have  been  introduced.  They  are 

•  Tactical  Intelligence  delay  (T  =  0,  1 ,  2,  •  •  •  9)  turns, 

•  Area  Intelligence  delay  (A  =  0,  1 ,  2,  •  •  •  9)  turns, 

•  Communications  delay  (C=  0,  1 , 2,  •  •  •  9)  turns, 

•  Feedback  delay  (F  =  0,  1 , 2,  •  •  •  9)  turns, 

•  Planning  levels  (specific,  general,  and  none  =  Ps,  Pg,  and  Pn),  and 

•  Mission  sets  consisting  of  one  to  four  individual  missions.  E.g.;  team  X  might 
be  assigned  win  right  and  don't  lose  the  crossovers  (X  =  M|_VjQg) 

The  levels  of  each  of  the  five  factors  may  be  distributed  among  players  each  game 
as  follows: 

•  Tactical  Intelligence  delay:  unrestricted  assignment  of  levels  to  players, 

•  Area  Intelligence  delay:  unrestricted  assignment  of  levels  to  players, 

•  Communications  delay:  unrestricted  assignment  of  levels  to  players, 

•  Feedback  delay:  unrestricted  assignment  of  levels  to  players, 

•  Planning  levels:  The  same  level  must  be  assigned  to  partners,  and 

•  Mission  assignments:  The  samp  mission  set  must  be  assigned  to  partners. 

In  order  to  get  a  feel  for  the  magnitude  of  the  experimental  design  options  assume 
all  factors  are  limited  to  three  levels  (the  four  delay  factors  are  limited  to  real  time,  one 
turn  delay,  or  no  information  (level  9)  and  Missions  are  limited  to  one  of  left  victory, 
right  victory,  or  overall  victory  then  consider  the  following: 

•  T0,  Ti .  T9  with  4  players  =  24  permutations 

•  Aq,  A-j  ,  Ag  with  4  players  =  24  permutations 

•  Cq,  C-j,  Cg  with  4  players  =  24  permutations 
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•  F0,  Fi  ,  Fg  with  4  players  =  24  permutations 

•  Ps,  Pg,  Pn  with  2  teams  =  6  permutations 

•  M  (  LV,  RV,  OV  )  with  2  teams=  6  permutations 

Combining  the  above  six  factors  and  their  three  levels  by  players  yields  a  wide 
spectrum  of  experimental  design  opportunities.  Allowing  more  than  three  levels  (e.g.; 
mission  sets  from  one  to  four  missions  may  be  assigned  to  the  two  teams  where  a 
mission  set  consists  of  combinations  of  the  eight  basic  missions)  would  increase  the 
number  of  permutations  significantly. 

Experience  gained  during  the  last  T 4  experiment  showed  that  while  game  play 
was  fairly  uncomplicated,  the  administration  of  the  game  by  the  controllers  was 
cumbersome,  confusing,  time  consuming,  and  error  prone.  A  computer  based  aid  for 
the  controllers  will  improve  this  facet  of  the  experiment.  Using  the  aid  will  permit 
improved  accuracy  and  allow  more  trials  during  a  given  time  period.  The  next  section 
discusses  this  computer  aid  in  depth. 


9 


T4  SIMULATION 


MOTIVATION 

The  initial  T4  experiment  was  conducted  as  part  of  the  OS4602 
course  in  the  fall  of  1990.  A  subset  of  the  students  were  assigned 
to  the  lead  group.  This  group  was  responsible  for  the  design, 
conduct,  analysis  and  reporting  the  results  of  the  experiment.  The 
remainder  of  the  students  served  as  subjects.  Each  trial  required 
for  subjects  assigned  to  two  teams.  Team  0  consisted  of  an  O  left 
and  right  player  while  team  X  consisted  of  a  X  left  and  X  right 
player. 

LESSONS  LEARNED 

In  retrospect  two  problem  areas  were  identified.  1 .)  The  scope 
of  the  experimental  design  was  overly  ambitious  given  the  class 
time  constraints  allotted  to  the  lead  group.  The  resulting  small 
number  of  trials  pre-preempted  findings  that  were  significant.  2.) 
Data  collection  errors  were  committed  by  the  lead  group  during  the 
conduct,  scoring,  and  data  reduction  phases  of  the  experiment. 

These  errors  may  have  led  the  lead  group  to  reach  an  erroneous 
conclusion. 

AUTOMATED  T4 

A  automated  version  of  the  team  T4  game  has  been  developed  to 
assist  lead  groups  conduct  more  trials  while  hopefully  eliminating 
the  data  errors  experienced  during  the  first  T4  experiment.  The 
automated  T4  allows  the  lead  group  to  configure  game  files  based  on 
the  experimental  design  prior  to  the  conduct  of  the  trials.  During 
the  trials  the  lead  group  receives  private  moves  from  each  of  the 
four  players  and  inputs  them  into  the  automated  T4  display  by 
clicking  a  mouse  on  the  appropriate  cell  on  the  electronic  game 
board.  After  the  four  moves  are  secretly  input  by  the  lead  group,  the 
program  then  evaluates  each  players  moves  based  on  the  game 
configuration  (for  example  the  amount  and  type  of  delay)  and  then 
prints  a  new  move  sheet  for  each  player  containing  this  filtered 
information,  The  individual  move  sheets  are  returned  to  each  player 
the  next  move  cycle  begins. 

Data  for  each  game  is  automatically  collected.  The  data  includes 
the  game  configuration  data,  a  history  of  each  move  by  player,  and 
game  scoring  by  TTTs,  and  mission  areas.  The  program  also  allows 
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the  lead  group  to  write  the  results  of  the  trials  to  a  file  in  matrix 
(spreadsheet)  format  for  further  data  reduction  and  analysis. 

T4  SIMULATION 

Developing  a  T4  simulation  was  a  natural  extension  to  the 
automated  T4  game.  The  automated  T4  game  already  contained  the 
software  to  record  legal  moves,  to  collect  a  wealth  of  game  data,  to 
score  the  games,  and  to  print  the  results  to  a  data  file.  All  that  was 
required  to  complete  the  T4  simulation  was  1.)  creation  of  method 
to  generate  controlled  game  moves  based  on  predefined  user 
specified  configurations  and  2.)  to  construct  a  mechanism  to  allow 
multiple  replications  of  these  games,  once  specified  without  user 
intervention. 

Advantages  of  T4  Simulation 

The  advantages  of  using  a  T4  simulation  include  1.)  testing  the 
performance  of  the  automated  game,  2.)  analyzing  relationships 
between  T4  configuration  factors  such  as  delays,  missions 
assignments,  player  tactics,  and  initial  wins,  and  3.)  permitting 
player  versus  computer  trials  which  helps  control  the  game  play 
variables  and  increases  the  number  of  trials  (  1  human  subject  can 
play  against  3  computer  players  instead  of  three  players. 

Status  of  T4  Simulation 

The  computer  versus  computer  version  of  the  simulation  is 
complete.  Item  1.)  testing  discussed  above  has  been  completed. 

The  Item  3.)  human  versus  computer  play  version  is  not  yet 
available.  Item  2.)  analysis  of  relationships  between  game 
configurations  factors  is  the  subject  of  the  remainder  of  this  paper. 

T4  SIMULATION  OVERVIEW 

The  current  version  of  the  T4  simulation  allows  the  user  to 
configure  play  of  a  trial  including  configuration  each  of  the  four 
computer  players.  Configuration  means  specifying  such  items  as 
team  mission  assignments,  delays  associated  with  each  player,  and 
player  tactics.  The  user  also  specifies  the  number  of  replications 
desired  for  each  trial  configuration  and  which  data  file  to  save  the 
data  collected  during  the  games.  Multiple  trial  configurations  may 
be  specified  for  a  given  simulation  run.  For  example  30  replications 
of  one  game  configuration  and  30  replications  of  one  or  more 


variations  to  this  configuration  may  be  specified  for  a  simulation 
run.  Once  the  user  starts  the  simulation  run  no  further  human 
intervention  is  required  until  the  replication  run  is  completed,  the 
T4  simulation  plays  the  games  according  to  the  predefined  user 
specifications  and  sends  the  results  to  user  specified  data  files. 
These  data  files  may  then  be  read  by  most  spreadsheets  or 
statistics  packages  for  further  data  reduction  under  macro  control. 
The  following  sections  discuss  the  details  of  T4  simulation  game 
configuration  including  creating  game  plans,  specification  of 
multiple  factor  levels,  description  of  the  data  that  is  automatically 
collected,  and  use  of  multiple  replications  to  simulate  T4  results. 

GAME  PLANS 

The  user  specifies  the  actions  of  the  four  computer  players  by 
creating  a  set  of  three  game  plans  for  each.  The  set  of  three  game 
plans  are  the  Regular  game  plan,  the  Crossover  game  plan,  and  the 
Cell  game  plan.  The  game  plans  enumerate  all  possible  next  plays 
for  the  player.  The  user,  by  assigning  point  values  to  these 
conditions,  controls  how  the  computer  player  plays  the  game. 

The  simulation  matches  the  actual  condition  of  the  game  board 
with  the  possible  next  moves  and  assigns  point  values  to  the  empty 
cells  (potential  next  moves).  First  the  game  board  is  matched 
against  the  regular  game  plan,  then  the  crossover  game  plan,  and 
finally  the  cell  game  plan,  the  scores  of  each  are  accumulated  in  the 
empty  cells.  The  cell  with  the  highest  point  count  is  chosen  for  the 
next  move.  Ties  are  randomly  broken.  A  more  complete  description 
of  each  of  the  three  types  of  game  plans  is  provided  below.  Also 
included  is  an  example  game  in  progress  to  illustrate  the  concepts. 

After  describing  game  plans,  the  other  user  specified  game 
configuration  are  discussed  followed  by  the  a  discussion  on  the 
automatic  data  collection  features  of  the  simulation.  We  will  then 
be  prepared  to  discuss  the  design  of  experiment  which  when 
executed  leads  to  a  better  understanding  of  the  relationships  of  the 
game  configuration  variables. 

REGULAR  GAME  PLAN 

The  regular  game  plan  regulates  the  automated  play  of  the 
computer  player  on  the  regular  (non-crossover)  portion  of  that 
player’s  side  of  the  game  board.  If  we  assume  we  are  describing 
Player  O  Left’s  game  plan  then  O  Left’s  regular  portion  of  the 
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playing  board  is  shown  as  unshaded  in  the  figure  below. 


T4  Game  Board 
Q  Left  0  Right 


"  j 

1 

1 

X  Left  X  Right 


The  simulation  first  sequentially  “looks”  at  all  eight  of  the  ways 
that  a  three  cell  regular  TTT  can  be  scored  as  shown  in  the  figure 
below. 


Eight  Regular  TTTs 


It  first  looks  at  the  3  horizontal  TTTs,  then  the  3  vertical  TTTs, 
then  the  2  diagonal  TTTs.  During  each  of  the  eight  looks  it  attempts 
to  match  the  contents  of  the  three  cells  in  the  look  with  a 
permutation  of  one  of  the  codes  in  the  regular  game  plan. 

A  regular  game  plan  is  a  table  which  enumerates  in  coded  form  all 
possible  next  moves  for  a  player  and  the  point  values  assigned  by 
the  user  to  that  move.  Player  O  Left’s  regular  game  plan  with 
sample  values  assigned  to  each  condition  is  shown  in  the  figure 
below. 


Regular  Game  Plan 
Player  O  Left 


oo- 

1  .000 

XX- 

500 

o-- 

100 

X-- 

50 

— 

1  0 

ox- 

0 

An  X  or  an  O  in  the  code  means  a  cell  has  an  X  or  an  O  in  it.  A  dash 
(*)  means  a  cell  is  blank.  As  previously  discussed,  the  program 
attempts  to  match  the  3  cells  in  the  current  look  with  permutations 
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of  the  codes  in  the  regular  game  plan.  For  example  the  simulation 
first  looks  at  the  ihree  cells  in  first  row  of  player  O  Left’s  game 
board  and  attempts  to  match  the  contents  with  a  permutation  of  the 
first  code  “OO-"  in  the  regular  game  plan.  A  match  occurs  if  the 
three  examined  ceils  contain  “OO”,  “O-O”,  or  “-00”.  In  other 
words,  a  match  occurs  if  the  three  ceils  contain  exactly  three  X’s 
and  a  blank  in  any  order.  The  remaining  codes  are  evaluated  in  the 
same  manner  as  shown  in  the  figure  below. 

Regular  Game  Plan  Definition 


Player  O  Left 

The  three  cells  contain 

(In  any  order)  exactly: 

OO- 

1  .000 

XX- 

500 

o-- 

1  00 

One  0  and  two  empty  cells 

X-- 

50 

One  X  and  two  empty  cells 

— 

1  0 

ox- 

0 

When  a  match  occurs  the  point  value  in  the  second  column  of  the 
game  plan  is  added  to  the  blank  cell.  Notice  that  each  code  has  at 
least  one  dash  (-)  in  it,  otherwise  a  next  move  for  that  code  would 
not  be  possible. 

The  simulation  then  steps  through  each  of  the  eight  TTT  looks, 
each  time  assigning  the  value  of  the  permuted  code  it  matches  to 
the  empty  cell.  Notice  that  if  an  empty  cell  is  in  the  center  of  the 
game  board  then  the  point  value  represents  the  sum  of  four  “looks” 
(two  diagonal,  one  horizontal,  and  one  vertical).  An  empty  corner 
cell  has  three  looks  and  an  empty  edge  cell  has  two  looks. 

Similar  procedures  are  used  to  add  the  values  in  the  crossover 
and  cell  game  plans  to  the  empty  cells.  But  first  the  following 
example  is  introduced  to  illustrated  the  procedure.  The  example 
game  is  joined  in  progress  and  the  simulation  is  evaluating  the  next 
move  for  O  left  using  the  regular  game  plan  shown  above.  Only  one 
side  of  the  game  board  is  used  to  evaluate  the  regular  game  plan:  in 
this  case  the  left  side. 
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O  Left's  Next  move 


O  Left  |  Q  Right 


X2 

X2 

XI 

02 

Ol 

02 

01 

XI 

X  Left  I  X  Right 


First  the  three  rows  are  evaluated.  The  top  row  (--X)  matches  a 
permutation  of  (X--)  in  the  regular  game  plan.  Therefore  50  points 
is  assigned  to  both  blank  cells  in  the  top  row.  The  middle  and 
bottom  rows  are  evaluated  in  the  same  manner  resulting  in  the 
interim  point  values  as  shown  in  the  following  figure. 


O  Left's  Next  move 


OLeft 

O  Right 

50 

50 

X2 

X2 

50 

XI 

50 

02 

Ol 

1000 

02 

Ol 

XI 

X  Left 

X  Right 

The  same  procedures  are  followed  in  matching  the  three  vertical 
columns  and  the  two  diagonals.  The  point  values  from  the  regular 
game  plan  is  added  to  the  values  already  in  the  empty  cell  (empty  in 
the  since  that  an  X  or  O  is  not  in  the  cell  and  thus  it  represents  a 
possible  next  move).  The  results  of  matching  and  adding  the  three 
vertical  rows  is  shown  in  the  figure  in  the  following  figure  on  the 
left.  The  point  values  found  from  matching  the  two  diagonals  is 
added  to  the  values  in  the  left  figure  and  shown  in  the  right  figure 
shown  below. 


O  Left's  Next  move 


O  Left’s  Next  move 


OLeft 

O  Right 

60 

50 

X2 

X2 

60 

XI 

50 

02 

01 

1010 

02 

Ol 

XI 

X  Left 

X  Right 

OLeft 

O  Right 

60 

50 

X2 

X2 

60 

XI 

50 

02 

Ol 

1510 

02 

Ol 

XI 

X  Left 

X  Right 

Based  on  the  results  of  assigning  point  values  to  empty  cells 
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\‘ 


based  on  the  regular  game  plan  only  the  next  move  selected  would  be 
the  lower  left  cell  (=1,510).  However  the  Crossover  and  Cell  game 
plans  also  contribute  to  the  point  values.  The  crossover  area  of  a  T4 
team  game  consists  of  all  the  cells  in  TTTs  that  cross  the 
centerline  as  shown  in  the  following  figures.  The  unshaded  portion 
of  the  figure  on  the  left  is  crossover  area.  The  twelve  crossover 
TTTs  for  player  O  Left  are  shown  on  the  right. 


Crossover  TTT  Area 


OLeft 


O  Right 


X  Left 


X  Right 


O  Left  Crossover  TTTs 

O  Left  |  O  Right 


SI 

3H 


(12 


X  Left 


X  Right 


Notice  that  while  crossover  TTTs  by  definition  extend  to  the  right 
side  of  the  game  board,  the  only  points  of  interest  to  O  Left  are 
those  on  the  left  side  of  the  board.  Points  on  the  right  side  of  the 
game  board  are  accumulated  using  the  game  plans  for  the  O  Right  and 
X  Right  players.  The  Crossover  game  plan  used  for  this  player  O  Left 
example  is  shown  below. 


Crossover  Game  Plan 


oo- 

1  .000 

XX- 

500 

0-- 

100 

X-- 

50 

• — 

1  0 

ox- 

0 

The  procedures  for  adding  point  values  to  the  empty  (non  X  or  O) 
cells  after  accounting  for  the  topology  of  the  12  crossover  TTTs  are 
the  same  as  those  used  to  calculate  the  regular  game  plan  points. 
Each  of  the  three  cells  in  the  twelve  crossover  TTTs  are  matched 
with  the  crossover  game  plan  and  empty  cells  on  the  left  side  of  the 
board  are  assigned  point  values  as  before. 

The  below  figure  on  the  left  shows  the  point  totals  after  the 
points  associated  with  the  three  left-most  horizontal  “looks”  have 
been  matched  and  accumulated  in  the  empty  cells.  The  figure  on  the 
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right  adds  the  points  from  the  right-most  horizontal  “looks”  to  the 
accumulated  totals 


O  Left's  Next  move 


OLeft 

0  Rinht 

60 

100 

X2 

X2 

60 

XI 

50 

02 

Ol 

1510 

02 

01 

Xi 

| 

X  Left 

X  Right 

O  Left's  Next  move 


OLeft 

O  Riaht 

60 

100 

X2  i 

X2 

60 

XI 

1050 

02 

Ol 

1510 

02 

Ol 

XI 

X  Left 

X  Right 

The  point  totals  after  adding  the  remainder  of  the  points  based  on 
the  vertical  and  diagonal  looks  are  as  shown  below.  Note  that  the 
two  vertical  looks  resulted  in  no  points  added  to  cells  because  they 
were  a  permutation  of  (XO-).  And  no  points  were  added  to  empty 
cells  based  on  the  right  most  two  diagonal  looks  because  there 
weren’t  any  empty  cells. 


O  Left's  Next  move 


OLeft 

O  Right 

60 

100 

X2 

X2 

60 

XI 

1160 

02 

Ol 

1510 

02 

Ol 

XI 

X  Left 

X  Right 

Finally  the  points  associated  with  the  Cell  Move  Plan  are  added  in. 
The  points  in  the  example  player  0  Left  Cell  move  plan  shown  in  the 
figure  below  left  are  simply  transferred  directly  to  any  empty  cells. 
It  may  appear  that  the  points  assigned  to  the  Cell  game  plan  in  this 
example  are  too  low  to  have  an  effect  on  the  outcome.  However  they 
may  serve  as  a  tie  breaker.  The  resultant  final  score  is  shown  on 
the  right. 


Cell  Game  Plan 


Plat 

fer  0  Left 

2 

3 

5 

1 

4 

5 

2 

3 

5 

O  Left's  Next  move 


OLeft 

O  Right 

62 

103 

X2 

X2 

61 

XI 

1165 

02 

Ol 

|| 

1512 

02 

Ol 

XI 

X  Left 

X  Right 

Because  the  lower  left  ceil  has  the  highest  point  total  it  is 
selected  as  player  O  Left’s  move  for  turn  three. 

The  same  procedures  are  used  with  each  player’s  unique  set  of 
three  game  plans  to  select  players  O  Right,  X  Left,  and  X  Right  next 
moves. 
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APPENDIX  B 


icotion  Sat 


GD 


Saue  Script  Load  Script  j  Nem  Data  File  New  Gome  Plan  Newt  Config^ 

(Data  lila],  Pin  1-1  Data  File 

. [PathNameJ.Mac  external  HD:T4:Run  1-1  Data  File 

(Gama  Plan],  Run  1  Gama  Plan 
. [PathNamel.Mac  external  HD:T4:Run  1  Gama  Plan 

....DDRR  30  [MO]  VL  VC  ••  “  (MX]  VL  VC  ”  **  [OLJ  000  [OR]  000  PCL]  000  [XR]  000 
....DDRR  30  [MO]  VL  VC  **  ”  (MX)  VL  VC  **  ••  [OL]  100  [OR]  100  [XL]  000  [XR]  000 


[MX]  VL  VC  **  •*  [OL]  100 
....DORR  30  [MO]  VL  VC  **  **  [MX]  VL  VC  **  "  [OL]  200 

....DDRR  30  [MO]  VL  VC  **  ”  [MX]  VL  VC  **  ••  [OL]  300 

....DDRR  30  [MO]  VL  VC  **  -  [MX]  VL  VC  '*  ••  [OL]  400 

....DDRR  30  (MO)  VL  VC  ~  **  [MX]  VL  VC  **  "  [OL]  500 

....DDRR  30  [MO]  VL  VC  ”  **  [MX]  VL  VC  ~  ”  [OL]  600 

....DDRR  30  [MO]  VL  VC  ~  **  [MX]  VL  VC  **  “  [OL]  700 

....DDRR  30  [MO]  VL  VC  *•  **  [MX]  VL  VC  ••  ••  [OL]  BOO 

....DDRR  30  [MO]  VL  VC  ••  ••  [MX]  VL  VC  **  **  [OL]  800 

[Data  lila].  Run  1-2  Data  File 

. [PathNameJ.Mac  external  HD:T4:Run  1-2  Data  File 

[Game  Plan],  Run  1  Game  Plan 
. [PathNamel.Mac  external  HO:T4:Run  1  Game  Plan 


[OR]  000 
[OR]  100 
[OR]  200 
[OR]  300 
[OR]  400 
[OR]  500 
[OR]  600 
[OR]  700 
[OR]  BOO 
[OR]  900 


[XL]  000 
[XL]  000 
[XL]  000 
[XL]  000 
[XL]  000 
[XL]  000 
[XL]  000 
[XL]  000 
[XL]  000 
[XL]  000 


[XR]  000 
[XR]  000 
[XR]  000 
[XR]  000 
[XR]  000 
[XR]  000 
[XR]  000 
[XR]  000 
[XR]  000 
[XR]  000 


TACTICAL 


'  Clear  j  Replication  S«t*Up  (  Go  ) 

Saue  Script  Load  Script  Neui  Data  File  Neui  Game  Plan  |  NeHt  Config 

[Data  file].  Run  1-2  Data  File 

. [PathNamel.Mac  external  HD:T4:Run  1-2  Data  File 

[Gama  Plan],  Run  1  Game  Plan 

. .[PathNameJ.Mac  external  HD:T4:Run  1  Game  Plan 


....DORR  30  [MO]  VL  VC  **  **  [MX]  VL  VC  **  **  [OL]  000 

....DDRR  30  [MO]  VL  VC  **  "  [MX]  VL  VC  •*  *'  [OL]  OtO 

....DDRR  30  [MO]  VL  VC  **  ”  [MX]  VL  VC  ”  **  [OL]  020 

....DDRR  30  [MO]  VL  VC  **  **  [MX]  VL  VC  **  ”  [OL]  030 

....DDRR  30  [MO]  VL  VC  ••  ”  [MX]  VL  VC  ••  ••  [OL]  040 

....DDRR  30  [MO]  VL  VC  **  **  [MX]  VL  VC  **  ”  [OL]  050 

....DDRR  30  [MO]  VL  VC  ”  “  [MX]  VL  VC  ••  ••  [OL]  060 

....DDRR  30  [MO]  VL  VC  **  **  [MX]  VL  VC  **  **  [OL]  070 

....DDRR  30  [MO,  VL  VC  **  **  [MX]  VL  VC  ~  ”  [OL]  OBO 

....ODRR  30  [MOj  VL  VC  “  "  [MX]  VL  VC  **  ”  [OL]  090 

[Data  file].  Run  1-3  Data  Fife 

. (PathNamel.Mac  external  HD:T4:Run  1-3  Data  File 

[Gama  Plan],  Run  1  Game  Plan 
. IPathNameLMac  external  HD:T4:Run  1  Game  Plan 


"  [OL]  000 
"  [OL]  OtO 
*’  [OL]  020 
”  [OL]  030 
**  [OL]  040 
**  [OL]  050 
**  [OL]  060 


[OR]  000  [XL] 
[OR]  010  [XL] 
[OR]  020  [XL] 
[OR]  030  [XL] 
[OR]  040  [XL] 
[OR]  050  [XL] 
[OR]  060  [XL] 
[OR]  070  [XL] 
[OR]  080  [XL] 
[OR]  090  [XL] 


000  [XR]  000 
000  [XR]  000 
000  [XR]  000 
000  [XR]  000 
000  [XR]  000 
000  [XR]  000 
000  [XR]  000 
000  [XR]  000 
000  [XR]  000 
000  [XR]  000 


AREA 


1 


Clear 


Replication  Set-Up 


GD 


Save  Script  Load  Script  Netu  Data  File  Neui  Game  Plan  Newt  Conflg 


[MX)  VL  VC  “  •*  [OL]  080 
[MX]  VL  VC  **  ■*  [OL]  090 


....DORR  30  [MO]  VL  VC  ”  * 

....DDRR  30  [MO]  VL  VC  **  • 

(Data  file].  Run  1-3  Data  File 

. [PathName],Mac  external  HD:T4:Run  1-3  Data  File 

[Game  Plan],  Run  1  Game  Plan 

. [PathNameJ.Mac  external  HD:T4:Run  1  Game  Plan 


[OR]  080 
[OR]  090 


[XL]  000  [XR]  000 
[XL]  000  [XR]  000 


....DDRR 

30 

[MO] 

VL  VC 

[MX]  VL  VC 

[OL]  000 

[OR] 

000 

[XL] 

000 

[XR] 

000 

....DDRR 

30 

[MO] 

VL  VC 

[MX]  VL  VC 

[OL]  001 

[OR] 

001 

[XL] 

000 

[XR] 

ooo 

....DDRR 

30 

[MO] 

VL  VC 

[MX]  VL  VC 

[OL]  002 

[OR] 

002 

[XL] 

000 

[XR] 

000 

....DDRR 

30 

[MO] 

VL  VC 

[MX]  VL  VC 

[OL]  003 

[OR] 

003 

[XL] 

000 

[XR] 

000 

....DDRR 

30 

[MO] 

VL  VC 

[MX]  VL  VC 

[OL]  004 

[OR] 

004 

[XL] 

000 

[XR] 

000 

....DDRR 

30 

[MO] 

VL  VC 

[MX]  VL  VC 

[OL]  005 

[OR] 

005 

[XL] 

000 

[XR] 

000 

...DDRH 

30 

[MO] 

VL  VC 

[MX]  VL  VC 

[OL]  006 

[OR] 

006 

[XL] 

000 

[XR] 

000 

....DDRR 

30 

[MO] 

VL  VC 

[MX]  VL  VC 

[OL]  007 

[OR] 

007 

[XL] 

000 

[XR] 

000 

....DDRR 

30 

[MO] 

VL  VC 

[MX]  VL  VC 

[OL]  008 

[OR] 

008 

[XL] 

000 

[XR] 

000 

....DDRR 

30 

[MO] 

VL  VC 

•* 

[MX]  VL  VC 

[OL]  009 

[OR] 

009 

[XL] 

000 

[XR] 

000 

[end] 
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APPENDIX  C 


AREA  DELAY 

MTB  >  aovoneway  cl-cIO 

ANALYSIS  OF  VARIANCE 

SOURCE  DF  SS  MS  F  p 

FACTOR  9  89.48  9.94  1.57  0.122 

ERROR  290  1831.07  6.31 

TOTAL  299  1920.55 

INDIVIDUAL  95  PCT  Cl'S  FOR  MEAN 
BASED  ON  POOLED  STDEV 

LEVEL  N  MEAN  STDEV  . ~  + . + . +  - 


A 

30 

0.167 

3.064  (~ 

. * . ) 

B 

30 

0.033 

2.442  - 

* . --) 

C 

30 

-0.067 

2.572  - 

—  * . ) 

D 

30 

1.067 

2.586 

( . * . ) 

E 

30 

0.967 

2.173 

( . * . ) 

F 

30 

0.733 

2.100 

( . * . ) 

G 

30 

0.867 

2.389 

( . * . ) 

H 

30 

1.600 

2.253 

( . * . 

1 

30 

1.100 

2.295 

( . * . ) 

J 

30 

0.000 

3.051  {— - 

—  * . ~) 

. +  — 

■ . + - + . . 

POOLED  STDEV  = 

2.513 

0.0  1.0 
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COMMUNICATIONS  DELAY 


MTB  >  aovoneway  cl  1-c20 


ANALYSIS  OF  VARIANCE 


SOURCE 

DF  SS 

MS  F 

P 

FACTOF 

1  9 

15.76 

1.75  0.22 

0.991 

ERROR 

290 

2306.17  7.95 

TOTAL 

299 

2321.93 

INDIVIDUAL  95 

PCT  Cl'S  FOR  MEAN 

BASED  ON  POOLED  STDEV 

LEVEL 

N 

MEAN 

STDEV  - . 

-  +  — - . + . +  — 

K 

30 

0.333 

2.783  ( . - 

-* . ) 

L 

30  - 

0.067 

3.248  ( . 

*  . ) 

M 

30 

-0.300 

3.164  ( . 

—  * . ) 

N 

30 

0.300 

2.926  (~ 

• . *- . ) 

0 

30 

0.167 

2.465  {— - 

- * . -) 

P 

30  - 

0.100 

3.111  ( . 

. * . ) 

Q 

30 

0.033 

2.659  ( . 

• . * . ) 

R 

30  - 

0.267 

2.318  ( . 

—  * . ) 

S 

30  - 

0.467 

2.738  ( . 

* . ) 

T 

30  - 

0.267 

2.638  ( - 

—  * . ) 

POOLED  STDEV  =  2.820 


■  + . 

-0.80 


•  + . 

-0.00 


0.80 
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TACTICAL  DELAY 


MTB  >  aovoneway  c21  -c30 


ANALYSIS  OF  VARIANCE 

SOURCE 

DF 

SS 

MS 

F 

P 

FACTOR 

9 

158.0 

17.6 

1.69 

0.092 

ERROR 

290 

3019.8 

10.4 

TOTAL 

299 

3177.8 

INDIVIDUAL  95  PCT  Cl'S  FOR  MEAN 
BASED  ON  POOLED  STDEV 

LEVEL  N  MEAN  STDEV  . + . —  + - + . +  - 


U 

30 

0.533 

2.751 

(- . ) 

V 

30 

1.867 

3.371 

( . * . ) 

W 

30 

1.633 

3.489 

( . -* . ) 

X 

30 

2.167 

2.902 

( - * . 1 

Y 

30 

2.000 

3.677 

( . -* . ) 

Z 

30 

1.767 

2.932 

< . * . ) 

AA 

30 

3.300 

2.855 

( . 

AB 

30 

1.733 

3.513 

( . * . ) 

AC 

30 

1.200 

3.624 

{ . * . ) 

AD 

30 

2.767 

2.979 

- +  - 

( . *~ 

. +  - . +  - . 

POOLED  STDEV  =  3.227  0.0  1.5  3.0  4.5 

MTB  >  NoOutfile. 
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APPENDIX  E 


MTB  >  anova  c3=c1  |c2 


Note:  A  = 

Type  of  Delay  , 

B  =  Level  of  Delay 

Factor 

Type  Levels  Values 

A 

fixed  3  1  2 

3 

B 

fixed  9  1  2 

3  4  5 

6  7  8  9 

Analysis  of  Variance  for  C 


Source 

DF 

SS 

MS 

F  P 

A 

2 

640.425 

320.212 

38.93  0.000 

B 

8 

69.217 

8.652 

1.05  0.395 

A*B 

16 

123.042 

7.690 

0.93  0.528 

Error 

783 

6440.700 

8.226 

Total 

809 

7273.384 

MTB  >  twoway  c3  cl  c2; 
SUBC>  mean  cl  c2. 

ANALYSIS  OF  VARIANCE  C 


SOURCE  DF  SS  MS 
A  2  640.42  320.21 

B  8  69.22  8.65 

INTERACTION  16  123.04  7.69 

ERROR  783  6440.70  8.23 

TOTAL  809  7273.38 

Individual  95%  Cl 

A  Mean  . + — -  + - +  - . -  + 

1  0.70 

2  -0.11  (—*-—) 

3  2.05  ( — *~~) 

. + . + . + - +  — - 

0.00  0.70  1.40  2.10 


1 


Individual  95%  Cl 


B  Mean  — +  - . h - + . + 

1  0.61  ( . ■— -* - ) 

2  0.42  ( . * - ) 

3  1.18  ( - * - ) 

4  1.04  ( - . ) 

5  0.80  ( - * - ) 

6  1.40  ( - * - ) 

7  1.02  ( - * - ) 

8  0.61  ( - * - ) 

9  0.83  ( - * - ) 

0.00  0.60  1.20  1.80 


MTB  >  NoOutfile. 
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